An FPGA-Based On-Device Reinforcement Learning Approach using Online Sequential Learning

DQN (Deep Q-Network) is a method to perform Q-learning for reinforcement learning using deep neural networks. DQNs require a large buffer and batch processing for an experience replay and rely on a backpropagation based iterative optimization, making them difficult to be implemented on resource-limited edge devices. In this paper, we propose a lightweight on-device reinforcement learning approach for low-cost FPGA devices. It exploits a recently proposed neural-network based on-device learning approach that does not rely on the backpropagation method but uses OS-ELM (Online Sequential Extreme Learning Machine) based training algorithm. In addition, we propose a combination of L2 regularization and spectral normalization for the on-device reinforcement learning so that output values of the neural network can be fit into a certain range and the reinforcement learning becomes stable. The proposed reinforcement learning approach is designed for PYNQ-Z1 board as a low-cost FPGA platform. The evaluation results using OpenAI Gym demonstrate that the proposed algorithm and its FPGA implementation complete a CartPole-v0 task 29.77x and 89.40x faster than a conventional DQN-based approach when the number of hidden-layer nodes is 64

Electromagnetic Form Factors of Nucleons with QCD Constraints Sytematic Study of the Space and Time-like Regions

Elastic electromagnetic form factors of nucleons are investigated both for the time-like and the space-like momentums under the condition that the QCD constraints are satisfied asymptotically. The unsubtracted dispersion relation with the superconvergence conditions are used as a realization of the QCD conditions. The experimental data are analyzed by using the dispersion formula and it is shown that the calculated form factors reproduce the experimental data reasonably well.Comment: 14 page

Velocity Structure of Jets in Coronal Hole

Velocity structures of jets in a coronal hole have been derived for the first time. Hinode observations revealed the existence of many bright points in coronal holes. They are loop-shaped and sometimes associated with coronal jets. Spectra obtained with the Extreme ultraviolet Imaging Spectrometer (EIS) on board Hinode are analyzed to infer Doppler velocity of bright loops and jets in a coronal hole of the north polar region. Elongated jets above bright loops are found to be blue-shifted by 30 km/s at maximum, while foot points of bright loops are red-shifted. Blue-shifts detected in coronal jets are interpreted as upflows produced by magnetic reconnection between emerging flux and the ambient field in the coronal hole.Comment: 11 pages, 7 figures, accepted for publication in PASJ Hinode special issu

Differences in Risk Factors for Rotator Cuff Tears between Elderly Patients and Young Patients

It has been unclear whether the risk factors for rotator cuff tears are the same at all ages or differ between young and older populations. In this study, we examined the risk factors for rotator cuff tears using classification and regression tree analysis as methods of nonlinear regression analysis. There were 65 patients in the rotator cuff tears group and 45 patients in the intact rotator cuff group. Classification and regression tree analysis was performed to predict rotator cuff tears. The target factor was rotator cuff tears; explanatory variables were age, sex, trauma, and critical shoulder angle≥35°. In the results of classification and regression tree analysis, the tree was divided at age 64. For patients aged≥64, the tree was divided at trauma. For patients aged<64, the tree was divided at critical shoulder angle≥35°. The odds ratio for critical shoulder angle≥35° was significant for all ages (5.89), and for patients aged<64 (10.3) while trauma was only a significant factor for patients aged≥64 (5.13). Age, trauma, and critical shoulder angle≥35° were related to rotator cuff tears in this study. However, these risk factors showed different trends according to age group, not a linear relationship

Pathogenesis of Frontotemporal Lobar Degeneration: Insights From Loss of Function Theory and Early Involvement of the Caudate Nucleus

Frontotemporal lobar degeneration (FTLD) is a group of clinically, pathologically and genetically heterogeneous neurodegenerative disorders that involve the frontal and temporal lobes. Behavioral variant frontotemporal dementia (bvFTD), semantic dementia (SD), and progressive non-fluent aphasia (PNFA) are three major clinical syndromes. TDP-43, FUS, and tau are three major pathogenetic proteins. In this review, we first discuss the loss-of-function mechanism of FTLD. We focus on FUS-associated pathogenesis in which FUS is linked to tau by regulating its alternative splicing machinery. Moreover, FUS is associated with abnormalities in post-synaptic formation, which can be an early disease marker of FTLD. Second, we discuss clinical and pathological aspects of FTLD. Recently, FTLD and amyotrophic lateral sclerosis (ALS) have been recognized as the same disease entity; indeed, nearly all sporadic ALS cases show TDP-43 pathology irrespective of FTD phenotype. Thus, investigating early structural and network changes in the FTLD/ALS continuum can be useful for developing early diagnostic markers of FTLD. MRI studies have revealed the involvement of the caudate nucleus and its anatomical networks in association with the early phase of behavioral/cognitive decline in FTLD/ALS. In particular, even ALS patients with normal cognition have shown a significant decrease in structural connectivity between the caudate head networks. In pathological studies, FTLD/ALS has shown striatal involvement of both efferent system components and glutamatergic inputs from the cerebral cortices even in ALS patients. Thus, the caudate nucleus may be primarily associated with behavioral abnormality and cognitive involvement in FTLD/ALS. Although several clinical trials have been conducted, there is still no therapy that can change the disease course in patients with FTLD. Therefore, there is an urgent need to establish a strategy for predominant sporadic FTLD cases

Bile Acids Induce Cdx2 Expression Through the Farnesoid X Receptor in Gastric Epithelial Cells

Clinical and experimental studies showed that the reflux of bile into the stomach contributes to the induction of intestinal metaplasia of the stomach and gastric carcinogenesis. Caudal-type homeobox 2 (Cdx2) plays a key role in the exhibition of intestinal phenotypes by regulating the expression of intestine-specific genes such as goblet-specific gene mucin 2 (MUC2). We investigated the involvement of the farnesoid X receptor (FXR), a nuclear receptor for bile acids, in the chenodeoxycholic acid (CDCA)-induced expression of Cdx2 and MUC2 in normal rat gastric epithelial cells (RGM-1 cells). RGM-1 cells were treated with CDCA or GW4064, an FXR agonist, in the presence or absence of guggulsterone, an FXR antagonist. CDCA induced dose-dependent expression of Cdx2 and MUC2 at both the mRNA and protein levels. The maximum stimulation of Cdx2 and MUC2 mRNA induced by CDCA was observed at 3 h and by 6 h, respectively. GW4064 also induced expression of these molecules. The effects of CDCA and GW4064 on expression of Cdx2 and MUC2 were abolished by guggulsterone. These findings suggest that bile acids may induce gastric intestinal metaplasia and carcinogenesis through the FXR